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THE 1948 ANNUAL HANDBOOK 


The 1948 edition of the ANNUAL HaAnp- 
Book is scheduled for publication on No- 
vember 1, 1947. Information concerning 
terms of admission, expenses, and scholar- 
ship aid is given for each college. The new 
edition is being distributed much earlier 
in the academic year than the previous edi- 
tion; it is hoped that the earlier publication 
date will make the HaNnpBook more gen- 
erally useful to school principals and guid- 


(Continued on page 18) 


"9 OF PROBLEM TYPES | 
oe COMPREHENSIVE MATHE 
TEST 


Multiple-Choice Type Found V 


By LyNNetTE B. PLUMLEE® 
Test Construction Department 


Within the past two years two important 
changes have been made in the Compre- 
hensive Mathematics Test. The first change 
was from an examination containing both 
traditional “demonstrative”-type problems 
and “answer-only” problems to an examina- 
tion containing the latter type exclusively. 
The second change was to an examina- 
tion composed entirely of “multiple-choice” 
problems. (Examples of these different prob- 
lem types are given in the following sec- 
tion.) The changes were made only after 
considerable investigation and deliberation. 


Factors which were taken into account in- 
clude: 


(a) Test reliability (agreement be- 
tween alternate forms of a test). 


(b) Test validity (agreement between 
test scores and achievement in 
college-mathematics courses ). 


(c) Accuracy, speed, and economy of 
scoring. 
(Continued on page 29) 


* For advice and general counsel in applying 
the results of this study to the Comprehensive 
Mathematics Test. acknowledgement is made to 
Professor S. S. Wilks, Consultant of the we 
Entrance Examination Board, and Professor J. R 
Kline, Chief Examiner in Mathematics. 
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The 1948 Annual Handbook 


(Continued from page 17) 


ance officers and to students who wish to 
gain admission to college in the fall of 1948. 
Other changes in the new edition in- 


clude new prefatory articles which describe - 


the Board’s series of entrance examinations, 
set forth the philosophy that underlies the 
Board’s testing programs, and list (with 
brief descriptions) those national and re- 
gional scholarship programs which involve 
use of the Board’s tests but which are not 
restricted to a single college or university. 
A greater degree of uniformity in the indi- 
vidual college statements has been achieved 
in this edition. Also, somewhat more spe- 
cific information is provided concerning the 
entrance tests required by each college, av- 
erage expenses at the college, and scholar- 
ships and loan funds. A new type of index 
groups the colleges geographically, and 
gives for each college the total undergradu- 
ate enrollment for 1946-1947. 

President-Emerita Ada Comstock Note- 
stein of Radcliffe College has again served as 
General Editor and as Chairman of the 
Handbook Committee. 

Copies of the Hanpsoox will be distrib- 
uted without charge to the colleges and 
schools most closely concerned with the 
Board's activities. Orders for additional cop- 
ies, at a price of $1.50, should be sent to 
the College Entrance Examination Board, 
425 West 117th Street, New York, N.Y. 
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KAIDEN-KAZANJIAN 
Proresson Epwarp S. Noyes 


Chairman of the College Entrance 
Examination Board, 1945- 


Edward S. Noyes, Associate Professor of 
English and Chairman of the Board of Ad- 
missions at Yale University, was elected 
Chairman of the College Entrance Exami- 
nation Board in October, 1945. For more 
than twenty-five years he had been closely 
associated with the work of the College 
Board, as Reader, Chief Reader, and from 
1939 to 1945 also as Chief Examiner in Eng- 
lish. During this period, in collaboration 
with Professor J. M. Stalnaker and Profes- 
sor W. M. Sale, he published studies of the 
comprehensive examination and of the new 
one-hour composition test, both of which he 
helped to develop. Before 1945 he served 
on the Committee on Examination Subjects 
and Requirements, on the Executive Com- 
mittee, and as Vice-Chairman of the Board. 

During Professor Noyes’ tenure as Chair- 
man, an important internal reorganization 
of the Board has been effected. Professor 
Noyes was a member of the committee 
which drew up the new ArTIcLEs oF As- 
SOCIATION AND By-Laws. 














NUMBER OF CANDIDATES REACHES NEW PEAK 


By Lepyarp R Tucker 
Department of Statistical Analysis 


The activity of the College Entrance Ex- 
amination Board has been at a record level 
during the present fiscal year. Sixty-five 
thousand candidates were examined in the 
regular college-entrance series; over 200,000 
test papers were scored. Both the number 
of candidates tested and the number of tests 
administered to these candidates increased 
about 40 per cent over the figures for 1946 
—itself a record year. 

Figure 1 shows the trend in number of 
candidates tested in the regular series dur- 
ing the last five years. The year 1943 is rep- 
resentative of the preceding five years. An 
increase in the number of candidates began 
in the war-years of 1943-1945, and has con- 
tinued at an accelerated pace during the 
last two years. The result of this trend is 
that the number of candidates tested is now 
double the number tested two years ago, 
and almost triple the pre-war level. 
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Fig. 1. 
Five-Year Trend in Number of Candidates 
Examined at Regular Series 


The Board’s work has also been charac- 
terized by striking seasonal variation. Fig- 
ure 2 shows the number of tests adminis- 
tered, scored, and reported for each of the 
regular series during the last two fiscal 





years.* Candidates at each series take from 
one to five tests, with an average of 3.3 
tests per candidate. This year the 58,000 can- 
didates who attended the morning session 
took two tests each: a verbal section, and a 
mathematical section of either the Scholas- 
tic Aptitude Test or the Comprehensive 
Mathematics Test. The 34,000 candidates 
who attended the afternoon sessions took 
one, two, or three tests each. A separate 
score was obtained and reported for each 
of the 200,000 test papers. About two-thirds 
of the work was in connection with the 
April examinations. 
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Fig. 2. 
Seasonal Variation in Tests Administered at Regular 
Series during the Last Two Fiscal Years 

Besides examining candidates in the reg- 
ular college-entrance series, the Board has 
examined during this year 40,000 candidates 
for the Naval College Training Programs, 
40,000 for Pepsi-Cola scholarships, 10,000 
veterans at special centers, and 10,000 can- 
didates for other programs (including en- 
trance examinations for the United States 
Naval and Military Academies). 


* The fiscal year of the College Board extends 
from October 1 to September 30 of the following 
year, and is named by the year in which it ends. 
Each fiscal year includes the December series in 
the preceding calendar year. 
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VALIDITY OF THE SPECIAL APTITUDE TEST FOR VETERANS 


By NorMAN FREDERIKSEN 
Research Department 


A study has just been completed in which 
the validity of the Special Aptitude Test for 
Veterans was investigated. Data for this 
study were obtained from several colleges 
of the University of Pennsylvania. The spe- 
cial veterans’ test was found to be almost 
as good a predictor of college grades as the 
regular tests of the College Board. 


THE VETERANS TEST 


The Special Aptitude Test for Veterans 
was prepared specifically for men and wom- 
en who were returning from war service. It 
was designed as a test which would em- 
phasize aptitude rather than achievement, 
in order that servicemen who have been out 
of school for several years might not be un- 
duly penalized. The first form of the test 
was designated SV, and the test has come to 
be known commonly as the SV test. 

The test consists of three one-hour sec- 
tions. The first section is verbal in content; 
scores on this section are comparable to 
verbal scores on the Scholastic Aptitude 
Test. Section Two is mathematical, and 
stresses mathematical manipulation and in- 
genuity rather than mastery of advanced 
subject matter. For Section Three the can- 
didate may choose Spatial Relations, Sci- 
ence, or Social Studies. The science section 
stresses common-sense informational ma- 
terial, and the social studies section requires 
the student to read with comprehension ma- 
terial from the social science field. The spa- 
tial relations section, testing ability to visu- 
alize spatially, was not investigated in this 
study. 

The special veterans’ test was adminis- 
tered at various member-colleges by regular 
college personnel. In addition, testing cen- 
ters were established in nine cities scat- 
tered across the country, where the test was 


[ 20} 


given every two weeks. In this way it was 
possible for veterans to be tested at various 
times and places. *Veterans who wished to 
be examined for college entrance soon after 
their discharge were thus not limited by the 
four-times-a-year schedule of the regular 
Board series. 


SOURCE OF THE DATA 


One of the first large-scale users of the 
Special Aptitude Test for Veterans was the 
University of Pennsylvania. Arrangements 
were made, through the cooperation of 
Sherman Oberly, Dean of Admissions, and 
Miles Murphy, Personnel Officer in the Col- 
lege of Arts and Sciences, to obtain the nec- 
essary information for a study of the test. 
Data were obtained for veteran students in 
the College of Arts and Sciences, the Whar- 
ton School of Finance and Commerce, the 
Towne Scientific School, and the Moore 
School of Electrical Engineering. Data from 
the last two schoois were combined, since 
the first-term freshman courses were essen- 
tially the same for many of the students in 
these schools. No information was obtained 
from the other colleges at the University of 
Pennsylvania because of the sma!l number 
of veteran students. 


PREDICTION OF GRADES IN THE 
COLLEGE OF ARTS AND SCIENCES 


The data obtained for veteran students in 
the College of Arts and Sciences included 
Verbal, Mathematical, and Social Studies 
scores on the Special Aptitude Test for 
Veterans, the high school quintile, number 
of years since high school graduation, and, 
as criteria, grades in all courses taken dur- 
ing the first term of the freshman year. From 
the course grades an unweighted average 
was computed, although at Pennsylvania 








average grades are not routinely calculated. 
The only course taken by a large proportion 
of the veteran freshmen was English 1. 

The correlations of the predictive meas- 
ures with English 1 grades and with first- 
term average are shown in Table 1. The 
best predictor of success in college, as meas- 
ured by first-term freshman average, is the 
verbal score (correlation of .49). The other 
two test scores have slightly lower validities, 
and high-school quintile has a validity cor- 
relation of .82. Number of years since high- 
school graduation has very little predictive 
value, even though there is considerable 
variability with respect to this factor.* Pre- 
diction of the first-term average grade can 
be improved somewhat by combining ver- 
bal and mathematical scores: the multiple 
correlation (not shown in the table) for 
this combination is .56. 


TABLE 1 


Correlations with Grades in 
the College of Arts and Sciences 











(N = 88) 
Correlation |Correlation 

Variable with First- with 
; Term Average} English 1 
SV—Verbal 49 52 
SV—Mathematical 39 31 
SV—Social Studies A2 45 
High-school quintile sa . a 
Years since high-school is 
_ graduation OT 13 





PREDICTION OF WHARTON SCHOOL GRADES 


All students in the Wharton School take 
the same set of required courses. Validity 
coefficients for this school are shown in 
Table 2. The table shows that the best pre- 
dictor of success is, in general, the verbal 
score, although the mathematics score is 
best for predicting grades in Accounting 1. 
High-school quintile has limited predictive 
value, and years since graduation again has 
little relation to college grades. 


® The standard deviation of the distribution of 
number of years since graduation was 2.7; the 
average number of years since graduation was 5.7. 


TABLE 2 


Correlations with Grades in 
the Wharton School 


(N = 152) 
| Correlation with- 
Variable Pol. 
Engl. 1 |Acct’g 1\Geog. 1| Sci. 1 
SV—Verbal AZ | 22 35 | 42 





SV—Mathematical 80 | 87 25 13 
SV—Social Studies 34 .26 32 32 


High-school quintile 19 27 27 30 
Years since high- 
school graduation| —.06 | .21 | 


PREDICTION OF GRADES IN SCIENCE SCHOOLS 


The criteria of success in the Moore 
School of Electrical Engineering and the 
Towne Scientific School were grades in 
several required courses: English 1, Mathe- 
matics N3, GE 101 (drawing), and Chem- 
istry 1. The validity of the predictive meas- 
ures for these schools is shown in Table 3. 
The N’s on which these correlations are 
based vary from 43 to 76. 


TABLE 3 


Correlations with Grades in 
the Two Science Schools 











(N = 48-76) 

ye | Correlation with— 
Variable 

Engl. 1} M&®- IGE 101\Chem.1 
SV—Verbal 52 | 08 | 28 21 
SV—Mathematical | .28 57 10 46 
SV—Science 28 .04 | 50 | .19 
High-school quintile} .25 52 03 | 55 
Years since high- 
school graduation| 42 | .09 17 |-.14 





Again the best predictor of English 1 
grades is the verbal score (correlation of 
52). Grades in Mathematics N38 are well 
predicted by the mathematical score (cor- 
relation of .57), while the drawing grade is 
best predicted by the science score (correla- 
tion of .50). The Chemistry grade, however, 
is more closely related to the high-school 
quintile (correlation of .55) than to any test 
score. For the veterans in the present study, 
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the number of years since graduation is 
most closely related to English 1 grade: 
those who have been out of school longest 
tend to make the best grades. The opposite 
tends slightly to be true for Chemistry 
grades. 


ROLE OF YEARS SINCE HIGH-SCHOOL 
GRADUATION 


The predominantly positive correlations 
between college grades and years since 
high-school graduation, shown in Tables 1, 
2, and 3, indicate that veterans who have 
been away from high school longest are not 
handicapped; as a matter of fact, there is a 
tendency—very slight—for veterans who 
have been out of school longest to make the 
best grades in college. To explain this fact 
one might conjecture that a selective factor 
is operating, such that the veterans who 
have been out of school longest tend not to 
go to college unless they have made un- 
usually good records in high school. The 
data in the first line of Table 4 bear on this 
hypothesis. Table 4 gives the correlations 
of years since graduation with high-school 
quintile and with veterans’ test scores. The 
correlations with high-school quintile are 
too low to ‘confirm the hypothesis that a 
selective factor is in operation. 


TABLE 4 
Correlations of Years Since High-School 
Graduation with High-School Quintile 
and Veterans’ Test Scores 


Correlations in the— 

















Variable College of 
Arts and |Wharton| Science 
Sciences | School | Schools 
High-school quintile | —.03 02 12 
SV—Verbal Pe: On 37 
SV—Mathematical = ST —.07 10 
SV—Social Studies 02 —.09 — 2 
SV—Science — —_—— 18 


It is possible that, at other universities, a 
higher correlation might be found between 
high-school achievement and years since 
graduation, because admissions policies 
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could easily influence this relationship. An 
admissions officer might, for example, re- 
fuse to admit the veterans who had been out 
of high school longest unless their high- 
school records were exceptionally good. 

The positive correlations of years since 
graduation with the verbal score suggest 
that higher verbal facility tends to be as- 
sociated with increasing experience or ma- 
turity. The predominantly negative correla- 
tions with the mathematical score suggest 
that mathematical ability scores tend to de- 
cline with lack of recent practice. 


CONCLUSIONS 


1, Although the samples are not large, 
the evidence indicates that the Special 
Aptitude Test for Veterans is valid for pre- 
dicting scholastic success as measured by 
first-term course grades; however, the valid- 
ity coefficients are in general somewhat 
lower than those usually obtained for the 
regular College Board tests. 

2. High-school quintile was found, in 
general, to have less predictive value than 
the test scores. Measures of high-school suc- 
cess usually have greater predictive value 
than was found in this study. The predictive 
value of the high-school quintile may have 
been lowered somewhat by the fact that, 
on the average, five or six years have elapsed 
since the graduation of these students from 
high school. It is also likely that the pre- 
dictive value of rank in class would be some- 
what greater than that of quintile-position, 
because of the coarseness of the latter meas- 
ure. 

8. Number of years since high-school 
graduation was found to have little rela- 
tionship to success in college; there was 
evidence that veterans who have been out 
of school longest are not handicapped with 
respect to ability to do college work. A 
slight tendency was found for better grades 
to be obtained by those out of school long- 
est. One might suppose that this slight 


tendency arises from self-selection—i.e., that 











students who have been out of high school 
for a long time tend not to come back to 
college unless they have been unusually 
successful in high school. This hypothesis of 


self-selection, however, is not borne out by 
the facts in the present case, because the 
correlations of years since graduation with 
high-school quintile are very low. 


SCHEDULE FOR NAVAL COLLEGE TRAINING PROGRAM, 1947-1948 


The Navy and Marine Corps are now 
starting their second annual selection of 
students for the Naval College Training Pro- 
gram. Winners in the competition will be 
enrolled in college NROTC units in the fall 
of 1948. Several thousand young men will be 
selected. The College Entrance Examination 
Board is again cooperating fully with the 
Navy Department in the preparation, ad- 
ministration, scoring, and reporting of the 
Navy College Aptitude Test, which all com- 
petitors are required to take as the first 
hurdle in the selection process. 'The whole 
schedule, in this second year, has been 
pushed back closer to the beginning of the 
academic year 1947-1948. 

Candidates can obtain a bulletin of in- 
formation and an application blank from 
schools and colleges in the States and Ter- 
ritories; these materials will be mailed from 
the Board during the third week of Septem- 
ber 1947. The Board will also send these ma- 
terials, on request, to individuals. A re- 
minder letter will be sent to schools and 
colleges about October 10-15. Enlisted men 
in the Navy and Marine Corps should apply 
through their commanding officers to the 
Bureau oF NAvAL PERSONNEL. Civilian 
candidates from schools and colleges must 
complete their application blanks and send 
them directly to the Nava ExaMmininc 
Section, CoLLEGE ENTRANCE EXAMINATION 
Boarp, P. O. Box 709, Princeton, N.J. A:- 
plications received at that address later than 
November 10, 1947 cannot be accepted. 

The Navy College Aptitude Test will be 
administered on December 13, 1947, at 


centers in every State and Territory. Each 
properly registered candidate will receive 
his ticket of admission before that date. The 
record sheets of the finalist candidates, who 
have qualified by test score for further con- 
sideration, will be mailed about January 
20, 1948 to the Offices of Naval Officer Pro- 
curement. These Offices will write directly 
to the finalists, who will then undergo physi- 
cal examinations and interviews. Announce- 
ments to unsuccessful candidates will be 
mailed about January 25. 

The Naval College Training Program re- 
mains the only nation-wide program of 
scholarship aid for college education that is 
financed completely by the federal govern- 
ment. The general plan for the program is 
substantially unchanged, and the benefits 
to the successful candidates remain the 
same. NROTC students will attend one of 
the 52 colleges having naval reserve units. 
Tuition, books, fees, and necessary uniforms 
will be provided by the Navy, and each 
student will receive a living allowance of 
$600 for each of the four years of attendance 
at college. Each student will then become a 
commissioned officer and will spend two 
years on active duty with the Navy or 
Marine Corps. 

By a recent decision of the Navy Depart- 
ment, no new students will be admitted to 
the Naval Aviation College Program in the 
fall of 1948; therefore no selection of new 
students for this program will be under- 
taken. About 400 winners from the first 
annual competition will be entering college 
under the NACP program in the fall of 1947, 
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VALIDITY OF THE GERMAN PLACEMENT TEST 
Relation between Test Scores and College Grades 


By Henry S. Dyer 
Harvard University 


College teachers of elementary foreign 
language have found from experience that 
the number of years an incoming freshman 
has studied a language in secondary school, 
and the quality of his work as evidenced by 
school grades, do not provide an adequate 
basis for deciding at what point the student 
should begin his study of the language in 
college. This situation is no reflection upon 
the instruction given by the preparatory 
schools. It is simply the result of a demo- 
cratic system in which the schools are left 
free to determine their own language cur- 
ricula and standards of grading. Neverthe- 
less, if college freshmen are to be placed in 
the language courses they are best equipped 
to handle, some common measure of cur- 
rent level of performance is indispensable; 
and this measure must have a close and 
ascertainabie relation to college standards. 
The C.E.E.B. foreign-language placement 
tests were devised to meet this need. The 
purpose of this paper is to present statistical 
evidence, obtained at Harvard, regarding 
the validity of the German Placement Test 
of the College Entrance Examination 
Board. 

One should clearly distinguish between 
the concept of a measure of current level of 
performance and the concept of a prediction 
of future performance. In the case of the 
C.E.E.B. German Placement Test, all we 
ask is that it tell us where the student is in 
the subject at the time he is to begin his 
college work. We do not ask the test to pre- 
dict the quality of the student’s work. If we 
can “place” him properly at the start, we 
assume that we have removed all the handi- 
caps that might prevent him from giving 
the best possible account of himself. 
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THE DATA 


An indication of the degree to which the 
C.E.E.B. German test measures “current 
level of performance” can be obtained by 
giving the test at a time when level of per- 
formance is known from some other source, 
which we call the criterion measure, and 
comparing the test scores with the criterion 
scores. In the present study the criterion 
scores were the final grades obtained in five 
types of elementary German courses, as fol- 
lows: 


German I, a first-semester course. 

German II, a second-semester course. 

German I-II, an intensive course which 
completes the work of the first two 
semesters in one semester. 

German III, a third-semester course. 

German III-IV, an intensive course 
which completes the work of the 
third and fourth semesters in one 
semester. 


Data were available from nine different 
classes: two classes in German I, one in 
German II, three in German I-II, one in 
German III, and two in German III-IV. The 
different courses were given variously in 
the spring, summer, and fall terms of 1945. 

The C.E.E.B. tests were administered ap- 
proximately two weeks before the final ex- 
amination in each course, but the results 
were not used in the computation of the 
final grades. It is believed that under these 
conditions the correlations obtained be- 
tween test scores and final grades provide a 
reasonably good index of the degree to 
which the test measures level of perform- 
ance in the reading of German. 





RESULTS 


Table 1 gives, for each course, the cor- 
relation between scores on the C.E.E.B. 
German Placement Test and final grades. 


TABLE 1 


Correlations between C.E.E.B. German 
Placement Test Scores and Final Grades 


No. of | Corre- 








Course Students| lation | r* 
German I (a) 114 64 Al 
German I (b) 43 A3 18 
German II 23 .78 61 
German I-II (a) 42 75 06 


German I-II (b) 41 .76 58 
German I-II (c) 21 77 9 
German III 52 .78 61 
German III-IV (a) 36 87 .76 
German III-IV (b)|} 27 ae koe 














It is evident from Table 1 that the cor- 
relation of test scores with final grades tends 
to be appreciably lower for the first- 
semester group than for the more advanced 
groups. This result is scarcely surprising. 
The C.E.E.B. German Test is aimed pri- 
marily at students who have completed at 
least two years of secondary-school work in 
German. In order to provide an optimum 
sampling of the reading ability of these 
and more advanced students, it seems rea- 
sonable to suppose that some sacrifice has 
had to be made in the number of questions 
placed on a difficulty-level suitable for 
students just beginning their study of Ger- 
man. 

One way of comprehending the “good- 
ness” of a correlation is to consider its 
square. The square of the correlation co- 
efficient in the present instance expresses 
the degree to which individual differences 
in final grades can be estimated from 
knowledge of the test scores alone. The 
squares of the correlation coefficients are 
shown in Table 1 in the column labeled 
“r?.” For all but the first-semester courses, 
the individual differences in final grades 
can be estimated to an extent greater than 


fifty per cent; or, to put it another way, the 
C.E.E.B. German Placement Test provides 
in a single hour of testing time a measure of 
performance which includes more than half 
of the factors which determine the final 
course-grades of students beyond the first- 
semester level. In the opinion of the writer, 
the results for the higher courses indicate 
a satisfactory degree of comparability be- 
tween test scores and grades. 


FACTORS NOT MEASURED BY THE TEST 


Factors which influence course grades 
but which are not measured by the test 
may well be accounted for by the following 
considerations: 

1. The German test purports to be solely 
a test of ability to read German; whereas 
the final grades represent, at least to some 
extent, a number of additional skills, such 
as those involved in German composition, 
which are not specifically sampled by the 
test. 

2. In certain individual cases there may 
have been an increase in the level of per- 
formance during the two weeks that elapsed 
between the C.E.E.B. test and the final ex- 
amination, since some students are prone 
to use just those two weeks to “cram for 
finals.” 

3. Certain of the final grades may have 
been affected by penalties inflicted during 
the semester for tardiness in the completion 
of assigned work. Such penalties often de- 
crease the final grade in such a way that it 
does not reflect the student’s true level of 
performance at the end of the term. 

The writer suggests that the unmeasured 
factors just described are not particularly 
relevant in determining the level of a 
student’s German reading performance; and 
that if it were possible to eliminate their 
effect on final grades, all but a negligible 
portion of individual differences in grades 
would be estimable from the test scores— 
except in the case of students at the first- 
semester level. 
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VALIDITIES OF SEPARATE PARTS OF THE TEST 


Under ordinary conditions, the separate 
sections of the German Test are not individ- 
ually timed; one hour is given for the entire 
test, and the student is left to his own de- 
vices to decide how he should apportion his 
time among the four sections. Since, within 
the hour allotted, a good many students do 
not finish the test and some may not reach 
Part IV at all, it was considered of some im- 
portance to determine under more rigor- 
ously controlled conditions the relative 
value of each part of the test. 

To investigate this matter, the test was 
given in certain courses with separate tim- 
ing of the individual parts, thereby insuring 
that each student had a chance to sample 
each part. The courses involved were Ger- 
man I (a), German I (b), German I-II (c), 
and German III. The correlations obtained 
between part scores and final grades give 
some idea of the relative validity of the dif- 
ferent parts for students of varying levels of 
competence. Table 2 gives the essential 
data. 


TABLE 2 


Correlations ‘between Part-Scores on the 
C.E.E.B. German Test and Final 
Grades in Various Courses 


TEST SECTION 
Part | Part | Part | Part |Total 
If | Ill | IV | Test 


Course 


German I (a) 
(N=114) | .29 | 48 | 54) 44 | 64 
German I (b) 
(N = 48) 15 | 35 | 36 | 22 | 48 
Ger. I-II (c) 
(N = 21) 71 | 5O | 69 | .72 | .77 
German III 
(N =52) 64 | .76 | 57 | .62 | .78 


Although the limitations of the data for- 
bid any general conclusions, they neverthe- 
less suggest that there is considerable differ- 
ence in the validity of the several parts for 
students of varying levels of competence. 
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Part I worked least well for first-semester 
students, second best for second- and third- 
semester students. Part IV worked best for 
second-semester students, third best for 
first- and third-semester students. A com- 
pa ison of the four “total test” correlations 
with those of the remaining five courses 
given in Table 1 seems to indicate that the 
timing of the individual parts has little if 
any effect on the over-all validity of the test. 


SUMMARY 


When the C.E.E.B. German Placement 
Test was given approximately two weeks 
before final examinations to students in 
elementary German courses at Harvard it 
was found that: 

1. The test scores provided a measure of 
reading performance sufficiently accurate 
to warrant their use for the placement of in- 
coming freshmen. 

2. The test is a somewhat less accurate 
measure of reading performance when used 
with first-semester students than it is when 
used with more advanced students. 

3. There is probably considerable differ- 
ence in the validity of the several parts of 
the test, some parts being more valid for 
students in less advanced classes, others 
more valid for students in later classes. 

4, Timing of the separate parts of the test 
does not appear to improve its over-all 
validity as a measure of reading perform- 
ance. 


A BINDER FOR THE "REVIEW" 


Many persons will find it convenient to 
file their copies of THe CoLtece Boarp 
Review in a loose-leaf binder. A well- 
constructed, attractive binder, designed 
especially for the Review, is obtainable 
from the College Entrance Examination 
Board (P.O. Box 592, Princeton, N.J.) at 
cost, for $2.00. This binder is 10 inches high 
and 9% inches wide; it will fit into nearly all 
book shelves. The “backbone” of the binder 
bears the title, Taz Cottece Boarp Review. 
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January test series added. In the academic 
year 1947-48 the College Board examina- 
tions will be given five times: in December, 
January, April, June and August. (For list of 
dates, see back cover.) There are several ad- 
vantages connected with a late January 
series. In the first place, it comes after the 
end of the first term, and so is a more natural 
testing time than December. In the second 
place, many colleges prefer to have their 
candidates examined earlier than April, es- 
pecially when only the Scholastic Aptitude 
Test is required for admission. In the third 
place, the January test meets the desires of 
students who must take their achievement 
tests in April, but prefer to take the SAT 
at a different time; because of the short in- 
terval between January and the April series, 
SAT scores made at either of these series 
should be closely comparable. 

Whether both the December and Janu- 
ary dates will be retained in 1948-49, or 
whether one or the other of them will be 
ome or shifted, will be decided at the 
October 1947 meeting of the Board. 


Intermediate Mathematics Test added to 
morning program. At its meeting in April 
1947 the Board voted to add to each series, 
beginning with the series of December 1947, 
an achievement test in mathematics for 
students with three years’ preparation in 
that subject. The new test, to be called the 
Intermediate Mathematics Test, will pre- 
suppose less training than does the Com- 
prehensive Mathematics Test, which will 
continue to cover four years of secondary- 
school mathematics. The Intermediate 
Mathematics Test will, however, call for 
more advanced mathematical training than 
the mathematics section of the Scholastic 
Aptitude Test, which stresses reasoning 
problems based on arithmetic and the rudi- 
ments of algebra and geometry. The three 
levels of mathematics tests will be given as 
alternatives in the morning session, fol- 
lowing the verbal section of the Scholastic 
Aptitude Test. 


Descriptions of the three morning pro- 
grams and sample questions from all of the 
tests are contained in the BULLETIN oF 
INFORMATION for 1947-48, obtainable from 
the College Entrance Examination Board 
on request. 


Greek and Italian tests will be offered. In 
April 1948, new one-hour achievement tests 
in Greek and Italian will be added to the 
present College Board battery. These ad- 
ditions will bring the number of afternoon 
achievement tests in the April series to a 
total of twelve, of which six will be foreign- 
language tests. Because of the relatively 
small number of candidates expected to 
register for Greek or Italian, it is probable 
that these two tests will be offered only at 
the specific examination centers where can- 
didates have indicated in advance that they 
wish to take them. The decision to add 
Greek and Italian was made by the Board 
at its meeting in October 1946, upon recom- 
mendations by the Committee on Examina- 
tions and the Executive Committee. No 
commitment was made to offer the tests 
at other times of the year or at the April 
series after 1948. 


English Examination for Foreign Students 
will be given in November 1947. The first 
regular edition of the English Examination 
for Foreign Students will be administered at 
overseas centers during the week of Novem- 
ber 9-16, 1947, and again during the week of 
February 29-March 7, 1948. The purpose of 
the examination, as described in the full- 
page article on page 7 of the last issue of the 
Review, is to aid in determining whether 
foreign applicants for admission to colleges 
in the United States have sufficient knowl- 
edge of English to enable them to undertake 
college work in this country. 

Regular examination centers will be dis- 
tributed all over the world, except in the 
United States. The exact date of the test at 
any particular center will be set, within 
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the time limits indicated above, by the local 
examination supervisor. In those centers 
where the local supervisors are willing, the 
examination may also be administered as 
often as once a month throughout the year, 
provided the number of candidates is suf- 
ficiently large. The examination fee at over- 
seas ce:.ters will be three dollars in the 
Western Hemisphere and ten dollars in the 
Eastern Hemisphere (United States cur- 
rency ). 

In addition to the regular overseas ad- 
ministrations, the examination will be made 
available to any institution for administra- 
tion to its own applicants at any time or 
place designated by that institution. In this 
way, foreign students who are already in 
this country may have an opportunity to 
take the test. Institutions using this service 
will be required to pay to the Board a fee 
of two dollars per candidate. 

Further information regarding the details 
of administration may be had upon request 
from the College Entrance Examination 
Board. It is expected that this examination 
will become a permanent feature of the 
Board’s program. 


West Coast office established. Plans have 
been completed for the establishment of a 
West Coast branch office of the Board. On 
October 1, 1947, the new office will be in 
operation as an information center and as a 
means of closer contact between the Board 
and the growing number of candidates and 
member colleges in the West. Dr. A. Glen- 
wood Walker has been appointed director 
of the new office, which is located at 2207 
Shattuck Ave., Berkeley 4, California. For 
the present, West Coast applications and 
fees should still be sent to P.O. Box 592, 
Princeton, N.J. 


Pepsi-Cola finalists to take Scholastic 
Aptitude Test. On January 24, 1948, more 
than 800 candidates for Pepsi-Cola scholar- 
ships and certificates of merit will report at 
Board centers to take the Scholastic Apti- 
tude Test. These candidates will be the top- 
scoring students among nearly 50,000 sen- 
iors in secondary schools throughout the 
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United States, Alaska, Hawaii, and Puerto 
Rico taking the Pepsi-Cola Scholarship Test 
on November 21, 1947. The use of a second 
examination marks a departure from the 
practice of previous years, when only the 
Pepsi-Cola Scholarship Test was employed. 


Further discussion of proposed merger of 
non-profit testing agencies. It was reported 
in the Spring 1947 issue of THe CoLLecE 
Boarp Review that the Board had appointed 
a committee to study the possibility of a 
merger of educational testing agencies, as 
proposed by the Committee on Testing of 
the Carnegie Foundation. The special com- 
mittee appointed by the Board issued a re- 
port which agreed with that of the Carnegie 
Committee in advocating an integration of 
non-profit testing groups (specifically, the 
testing agencies of the College Board, the 
American Council on Education, and the 
Carnegie Foundation), but which differed 
from the Carnegie Committee’s recommen- 
dations in regard to the structure, financing, 
and autonomy of the proposed new organi- 
zation. The proposal provides for the contin- 
uation of the College Entrance Examination 
Board as a separate educational associa- 
tion, but advocates that a central testing 
agency be established to handle the ex- 
aminations of the Board and of other organ- 
izations concerned with educational meas- 
urement, After discussion of the plan the 
Board at its April 1947 meeting voted 
unanimously to adopt the report of its 
special committee in principle, as a basis for 
negotiation with the other agencies con- 
cerned. The merger plan is at present under 
active consideration by the parties to the 
proposed new organization, and will be the 
subject of further discussion at the October 
1947 meeting of the Board. Future develop- 
ments concerning this important proposal 
will be reported in succeeding issues of the 
REVIEW. 


Fourteen new member institutions. The 
College Entrance Examination Board ex- 
tends a most sincere welcome to the four- 
teen colleges and universities admitted to 











membership in April. The new members 
are Boston University, Chestnut Hill Col- 
lege, Coe College, Muhlenberg College, 
Newcomb College of Tulane University, 
Oberlin College, Pomona College, Randolph- 
Macon Woman's College, Rosemont Col- 


lege, University of Notre Dame, Ursinus 
College, Washington and Jefferson College, 
Whitman College, and Wilson College. We 
are prouci to add these names to the College 
Board roster, which now lists sixty-nine 
member institutions. 


Comparison of Problem Types in the Comprehensive Mathematics Test 
(Continued from page 17) 


(d) Subject-matter coverage (ade- 
quacy with which the test covers 
the desired material). 


EXAMPLES OF PROBLEM TYPES 


Following are examples of the problem 
types mentioned above: 


(1) “Demonstrative” type— 

Two guns are fired at the same instant 
and an observer hears the report of 
the guns 3 seconds and 5 seconds later. 
The angle at the observer's eye sub- 
tended by the distance between the 
guns is 38° 43’. How many feet apart 
are the guns? (Assume that sound 
travels 1100 feet per second. ) ° 


This type of problem is designed to meas- 
ure performance of a connected series of 
operations. The amount of credit received 
by the candidate for each problem depends 
upon the reader’s judgment of the com- 
pleteness and accuracy of the candidate’s 
solution. 


(2) “Answer-only” type— 


A car travels from N to P at an average 
rate of r miles per hour and returns 
from P to N at an average rate of t 
miles per hour. What is the car’s aver- 
age rate for the whole trip? 


————m.p.h.+ 


In this type of problem, the candidate is 
given full credit if his final answer is cor- 
rect, and no credit if it is wrong. 


* The correct answer is 3580 ft. (using 4-place 
tables ). 


+ The correct answer is art 
r 


(3) “Multiple-choice” type— 

In this type of problem answer-options 
are given. The incorrect answer-options are 
based on the most popular wrong methods 
of solution adopted by students. As an ex- 
ample of the “multiple-choice” type, the 
foregoing “answer-only” problem is present- 
ed below with answer-options. 


A car travels from N to P at an average 
rate of r miles per hour and returns 
from P to N at an average rate of t 
miles per hour. What is the car's aver- 
age rate (in miles per hour) for the 
whole trip?* 


r+t 2 r+t 
ae re ne 
2rt 


A summary of the evidence, which led 
finally to the adoption of the multiple-choice 
type of problem, is: presented in the fol- 
lowing paragraphs. 


A COMPARISON OF MULTIPLE-CHOICE 
AND ANSWER-ONLY TYPES 


In a special study made in 1942, 80 math- 
ematics problems were prepared both in 
multiple-choice form and in answer-only 
form. The first 40 questions were adminis- 
tered to Group A (283 college students) in 
multiple-choice form and to Group B (282 
college students) in answer-only form. The 
last 40 questions were administered to 
Group A in answer-only form and to Group 
B in multiple-choice form. Coefficients of 


* The correct answer is —_ (E). The most 
popular wrong method of solution, a simple aver- 
age, leads to option (C). 
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reliability and validity for each answer-type 
were calculated, separately for each group, 
as shown in Table 1. The comparison, for 
each group, of the coefficients for multiple- 
choice with the corresponding coefficients 
for answer-only, shows quite clearly that the 
two problem types were practically equally 
reliable, and that they predicted rank-stand- 
ing in the first-year mathematics course 
equally well. 


TABLE 1 


Comparison of Multiple-Choice 
and Answer-Only Types 


Multiple- | Answer- 
Choice Only 
Form Form 
Reliability Coefficient: ° 

Cogut A -..:csoi torn .74 79 
CRED nonssseiscneiss .76 75 

Validity Coefficient: 
6 OF  Waeninc ee 64 .60 
sem RE. ois c..csous 57 58 





“ANSWER-ONLY FORM SUPERIOR 
TO “DEMONSTRATIVE FORM 


In a study of the April 1944 Comprehen- 
sive Mathematics Test, the reliability of the 
section of answer-only problems was found 
to be considerably higher than that of the 
section of demonstrative problems, for com- 
parable testing time. The reliability coeffi- 
cient of the answer-only section (computed 
for 553 candidates) was .92; that of the 
demonstrative section (computed for 552 
candidates) was .75. It would be expected, 
therefore, that the reliability of the total test 
would have been increased if the section of 
demonstrative problems had been replaced 
by a second section of answer-only prob- 
lems. 

In a study made several years ago at two 
universities, a set of answer-only problems 
and an equally timed set of demonstrative 
problems were compared as to their value 
for predicting success in college mathemat- 


* The reliability coefficient of each answer-type 
is the correlation between the odd- and even-num- 
bered problems, corrected by the Spearman-Brown 
formula. 

+ The validity coefficient is the correlation be- 
tween scores on each answer-form and rank-stand- 
ing in a first-year college mathematics course. 
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ics courses. Average correlations between 
course-grade and scores on each section of 
the test, for nine mathematics classes at Uni- 
versity X, and three mathematics classes at 
University Y, are shown in Table 2, below. 
The number of students per class ranged 
from 17 to 185 at University X and from 
28 to 158 at University Y, with an average 
of 65. The comparisons in Table 2 indicate 
that, for a given testing time, the answer- 
only section yields higher correlations with 
grades in college mathematics than does the 
demonstrative section. 


TABLE 2 


Predictive Validity of Answer-Only and 
Demonstrative Sections of the 
Comprehensive Mathematics Test 


Average Correlation 


University Answer-Only | Demonstrative 
Section Section 
University X 
(9 classes ) 44 37 
University Y 
(8 classes } 44 38 


SCORING 


Of the three answer-forms discussed in 
this article, the multiple-choice can, of 
course, be most quickly and accurately 
scored. In scoring a demonstrative problem, 
the candidate’s solution is given credit ac- 
cording to the completeness and accuracy of 
his method. However, the variety of methods 
used by different candidates in solving a 
demonstrative problem makes it impossible 
to compare accurately the quality of various 
solutions. The judgment as to equivalence 
of work by various methods is necessarily 
largely subjective. 

Problems involved in scoring also consti- 
tute a major disadvantage of the answer- 
only type. Methods of solution which are 
essentially correct may not be given credit 
because of minor computational or tran- 
scription errors. Furthermore, a candidate 
who carries his work beyond the minimum 
stage required. for credit receives no extra 
credit to compensate for the time he loses, 
and he has additional opportunity for mak- 
ing computational errors. It is impossible to 
establish rigidly consistent scoring pro- 








cedures for the answer-only test which do 
not work injustices in some cases. 

The multiple-choice form, on the other 
hand, definitely establishes the stage of solu- 
tion that must be reached by all candidates. 
Minor computational errors may be cor- 
rected by the candidate. Wrong methods 
are likely to lead to one of the wrong an- 
swer-options. The burden of decision— 
which constitutes part of the test—is placed 
upon the candidate, rather than upon the 
scorer, 

A practical factor requiring some con- 
sideration is the fact that accurate clerical 
workers may be used to score a multiple- 
choice test; whereas the scoring of an an- 
swer-only test requires persons with suffi- 
cient knowledge of mathematics to de- 
termine whether or not the recorded answer 
is equivalent to the answer on the key. The 
scoring of demonstrative problems requires 
still more highly qualified persons, who un- 
derstand the subject-matter sufficiently to 
assay the nature and seriousness of the can- 
didate’s errors. 

The approximate time required for scor- 
ing and checking a ninety-minute test in 
each of the three answer-forms is indicated 
in Table 3. Scoring and checking obviously 
take much longer for the demonstrative type 
of examination than for the multiple-choice. 


TABLE 3 


Approximate Time Required for Scoring 
Different Types of Mathematics Problems 





Time Required 
for Scoring and 
Checking 
100 Papers 


Answer Form | Number of 





~ Multiple-Choice 2 hours 
Answer-Only 12 hours 
Demonstrative 45 hours 


SUBJECT-MATTER COVERAGE 


The foregoing evidence appears sufficient 
reason for preferring the multiple-choice to 
the answer-only or demonstrative forms for 
the Comprehensive Mathematics Test. The 
exclusive use of multiple-choice problems 
does not, however, mean that certain subject 
matter may now be omitted from the sec- 
ondary school teaching of mathematics. 


Most problems which can be put into an- 
swer-only form can also be used in multiple- 
choice form; actually, some problems are 
more adaptable to the multiple-choice than 
to the answer-only form. Furthermore, the 
essential steps of the demonstrative prob- 
lem can be tested in separate short problems 
of the multiple-choice variety. Separate 
short problems force the candidate to give 
evidence of his competence in each specific 
type of mathematical work. For example, a 
rapid computer can frequently avoid loga- 
rithms entirely in solving a demonstrative 
problem; but he must know how to use 
logarithms, in order to answer correctly a 
short problem aimed at testing this specific 
knowledge. In addition, a wider range of 
abilities can be measured by numerous short 
problems than by a much smaller number 
of long problems. 


SUMMARY 


In consideration of the facts summarized 
below, the Comprehensive Mathematics Test 
has been changed to multiple-choice form: 

1. The multiple-choice test is equally re- 
liable and equally useful for predictive 
purposes as an answer-only test ad- 
ministered with the same time limit. 
(Consult Table 1.) 

2. A test composed of short problems, 
whether in multiple-choice or in an- 
swer-only form, is more reliable and 
more valid than a demonstrative test 
administered with the same time limit. 
(Consult page 30 and Tables 1-2.) 

3. Multiple-choice problems can be more 
easily and more accurately scored 
than either of the other two answer- 
forms. (Consult Table 3.) 

4. Subject matter may be covered about 
equally well in multiple-choice and an- 
swer-only tests, and more adequately 
in these tests than in the demonstrative 
test. (Consult page 31.) 

It should be noted that the studies de- 
scribed in this article aimed to determine 
the best type of mathematics test for pur- 
poses of prediction. No implication is in- 
tended regarding the suitability of mul- 
tiple-choice tests for diagnosing individual 
students’ specific difficulties in mathematics. 
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DATES OF COLLEGE-ENTRANCE EXAMINATIONS 


The College Entrance Examination Board has set the following 
schedule for its series of entrance examinations: 


Saturday, December 6, 1947 
Saturday, January 24, 1948 
Saturday, April 3, 1948 
Saturday, June 5, 1948 
Wednesday, August 18, 1948 


The complete series of examinations will be given on each examination 
date, except that the Greek and Italian examinations are scheduled 

only for April 3, 1948. 
| Applicants are urgently advised to obtain and complete their ap- 
plication forms early. The normal closing dates for the receipt of 
applications in Princeton, N.J. are as follows: for candidates east of 
_ the Mississippi River or on either bank of the Mississippi, three weeks 
before the date of the examination; for candidates west of the Missis- 
sippi River or in Canada, Mexico, or the West Indies, four weeks before 
the date of the examination; and for candidates outside of the United 
| States, Canada, Mexico, and the West Indies, seven weeks before the 
| date of the examination. Applications will continue to be accepted 
until a week before the examination date, but a penalty fee of three 
dollars (which should accompany the application ) will be charged for 
applications received later than the normal closing date. No applica- 
| tions.received in Princeton later than one week before the examination 
date will be considered. 

The Board’s BULLETIN OF INFoRMATION, obtainable on request, gives 
detailed information concerning the examination-schedule, applica- 
tions, fees, examination centers, and the general nature of the ex- 
| aminations. 








DATE OF THE NROTC APTITUDE TEST 


Civilian students from schools or colleges who wish to enroll in 
| college NROTC units in the fall of 1948 must take the Navy College 
| Aptitude Test. The date set for this examination is: 

Saturday, December 13, 1947 

| Applications must be received in Princeton, N.J., no later than 
November 10, 1947. 


For further particulars see the article on page 23 in this issue of 
Tue CoLiece Boarp Review. 





